When to Say What and How: Adapting the Elaborateness and Indirectness of Spoken Dialogue Systems

With the aim of designing a spoken dialogue system which has the ability to adapt to the user's communication idiosyncrasies, we investigate whether it is possible to carry over insights from the usage of communication styles in human-human interaction to human-computer interaction. In an extensive literature review, it is demonstrated that communication styles play an important role in human communication. Using a multi-lingual data set, we show that there is a significant correlation between the communication style of the system and the preceding communication style of the user. This is why two components that extend the standard architecture of spoken dialogue systems are presented: 1) a communication style classifier that automatically identifies the user communication style and 2) a communication style selection module that selects an appropriate system communication style. We consider the communication styles elaborateness and indirectness as it has been shown that they influence the user's satisfaction and the user's perception of a dialogue. We present a neural classification approach based on supervised learning for each task. Neural networks are trained and evaluated with features that can be automatically derived during an ongoing interaction in every spoken dialogue system. It is shown that both components yield solid results and outperform the baseline in form of a majority-class classifier

IQ-adaptive statistical dialogue management using Gaussian processes

Adapting a Spoken Dialogue System to the user's satisfaction is supposed to result in more successful dialogues. In this thesis, Gaussian processes are used to model a policy for a statistical Spoken Dialogue System and the Interaction Quality (IQ) metric which is a measure for the user's satisfaction is used to train this policy. As the policy decides which actions are taken next at a particular point, the dialogue flow is thus adapted to the IQ. Afterwards, it is investigated whether the incorporation of the IQ metric is beneficial. Therefore, different learning strategies with and without the IQ metric are used to train different policies. Then, the performance of all trained policies is evaluated regarding dialogue completion, task success, the average length of a dialogue and the average IQ value at the end of a dialogue

Automatic modification of communication style in dialogue management

Comunicació presentada a: INLG 2016 Workshop on Computational Creativity and Natural Language Generation, celebrat a Edinburgh, Escòcia, del 5 al 8 de setembre de 2016.In task-oriented dialogues, there is often only one right answer the system can give. However, a lack of variation can seem repetitive and unnatural. Humans change the way they express something, e.g. by being more or less concise. We aim to approximate this ability by automatically varying the level of verbosity and directness of a given system action. In this work, we illustrate how verbosity and directness may be utilised in adaptive dialogue management and present different approaches to automatically generate varying levels of verbosity and directness for given system actions. Thereby, new and unforeseen system actions can be created dynamically.This paper is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 645012

A social companion and conversation partner for elderly

Comunicació presentada a: 8th International Workshop On Spoken Dialogue Systems (IWSDS), celebrat del 6 al 9 de juny a Farmington, Estats Units.In this work, we present the development and evaluation of a social companion and conversation partner for the special user group of elderly.With the aim of designing a user-adaptive system, we responded to the desires of the elderly which have been identified during various interviews and created a companion which talks and listens to the elderly users. Moreover, we conducted a user study with a small group of retired seniors living at home or in a nursing home. The results show that our companion and its dialogue were perceived very positively and that a social companion and conversation partner is indeed in demand by lonely seniors.This work is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 645012

Automatic modification of communication style in dialogue management

Comunicació presentada a: INLG 2016 Workshop on Computational Creativity and Natural Language Generation, celebrat a Edinburgh, Escòcia, del 5 al 8 de setembre de 2016.In task-oriented dialogues, there is often only one right answer the system can give. However, a lack of variation can seem repetitive and unnatural. Humans change the way they express something, e.g. by being more or less concise. We aim to approximate this ability by automatically varying the level of verbosity and directness of a given system action. In this work, we illustrate how verbosity and directness may be utilised in adaptive dialogue management and present different approaches to automatically generate varying levels of verbosity and directness for given system actions. Thereby, new and unforeseen system actions can be created dynamically.This paper is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 645012

The next step: intelligent digital assistance for clinical operating rooms

With the emergence of new technologies, the surgical working environment becomes increasingly complex and comprises many medical devices that have to be taken cared of. However, the goal is to reduce the workload of the surgical team to allow them to fully focus on the actual surgical procedure. Therefore, new strategies are needed to keep the working environment manageable. Existing research projects in the field of intelligent medical environments mostly concentrate on workflow modeling or single smart features rather than building up a complete intelligent environment. In this article, we present the concept of intelligent digital assistance for clinical operating rooms (IDACO), providing the surgeon assistance in many different situations before and during an ongoing procedure using natural spoken language. The speech interface enables the surgeon to concentrate on the surgery and control the technical environment at the same time, without taking care of how to interact with the system. Furthermore, the system observes the context of the surgery and controls several devices autonomously at the appropriate time during the procedure

Estimating User Communication Styles for Spoken Dialogue Systems: Data

We present a neural network approach to estimate the communication style of spoken interaction, namely the stylistic variations elaborateness and directness, and investigate which type of input features to the estimator are necessary to achive good performance. First, we describe our annotated corpus of recordings in the health care domain and analyse the corpus statistics in terms of agreement, correlation and reliability of the ratings. We use this corpus to estimate the elaborateness and the directness of each utterance. We test different feature sets consisting of dialogue act features, grammatical features and linguistic features as input for our classifier and perform classification in two and three classes. Our classifiers use only features that can be automatically derived during an ongoing interaction in any spoken dialogue system without any prior annotation. Our results show that the elaborateness can be classified by only using the dialogue act and the amount of words contained in the corresponding utterance. The directness is a more difficult classification task and additional linguistic features in form of word embeddings improve the classification results. Afterwards, we run a comparison with a support vector machine and a recurrent neural network classifier

A social companion and conversation partner for elderly

Comunicació presentada a: 8th International Workshop On Spoken Dialogue Systems (IWSDS), celebrat del 6 al 9 de juny a Farmington, Estats Units.In this work, we present the development and evaluation of a social companion and conversation partner for the special user group of elderly.With the aim of designing a user-adaptive system, we responded to the desires of the elderly which have been identified during various interviews and created a companion which talks and listens to the elderly users. Moreover, we conducted a user study with a small group of retired seniors living at home or in a nursing home. The results show that our companion and its dialogue were perceived very positively and that a social companion and conversation partner is indeed in demand by lonely seniors.This work is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 645012

Cultural communication idiosyncrasies in human-computer interaction

Comunicació presentada a: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue; celebrada del 13 al 15 de setembre de 2016 a Los Angeles, USAIn this work, we investigate whether the cultural idiosyncrasies found in human human interaction may be transferred to human-computer interaction. With the aim of designing a culture-sensitive dialogue system, we designed a user study creating a dialogue in a domain that has the potential capacity to reveal cultural differences. The dialogue contains different options for the system output according to cultural differences. We conducted a survey among Germans and Japanese to investigate whether the supposed differences may be applied in human-computer interaction. Our results show that there are indeed differences, but not all results are consistent with the cultural models.This work is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 645012. This research and development work was also supported by the MIC/SCOPE #152307004